Tags: #tech, #llm

Backlinks: Local AI

Created: 2025-05-18 02:05
Last edited: 2026-01-31 00:33

LLAMA CPP

Setup

(based on vicuna-tools/vicuna-installation-guide but using another LLAMA repo)

• $ git clone https://github.com/ggerganov/llama.cpp.git
• $ cd llama.cpp
• $ make -j
• $ cd models
• $ wget -c https://huggingface.co/CRD716/ggml-vicuna-1.1-quantized/resolve/main/ggml-vicuna-13B-1.1-q5_1.bin

Use

• $ ./server -t 12 -m models/ggml-vicuna-13B-1.1-q5_1.bin
• open 127.0.0.1:8080